Skip to main content
Version: 1.0.0

operations

Operations​

1. groupBy

Syntax:  groupBy(fieldsArr, reducers)
parameters:
 - fieldsArr:
   required: true
   type: Array of string
   descriptions: Array containing the name of dimensions using which groupBy should happen.
 - reducers:
   required: false
   type: Array of Array,
   default: [],
   description : A simple array of (array pair) whose 0th index is the variable name and 1st
                 index is the name of the Aggregation function.

Groups the data using particular dimensions by reducing measures. It expects a list of dimensions using which it projects the DataModel and perform aggregations to reduce the duplicate tuples.

DataModel by default provides Aggregation Functions to aggregate grouped measure value.

Returns: DataModel (Returns a new DataModel instance after performing the groupBy)

const Datamodel = muze.DataModel;
const data = [
  {
    Maker: "chevrolet",
    Name: "chevrolet chevelle malibu",
    Miles_per_Gallon: 18,
    Cylinders: 8,
    Displacement: 307,
    Horsepower: 130,
    Weight_in_lbs: 3504,
    Acceleration: 12,
    Year: "1970-01-01",
    Origin: "USA",
  },
  {
    Maker: "buick",
    Name: "buick skylark 320",
    Miles_per_Gallon: 15,
    Cylinders: 8,
    Displacement: -350,
    Horsepower: 165,
    Weight_in_lbs: 3693,
    Acceleration: 11.5,
    Year: "1970-01-01",
    Origin: "USA",
  },
  // ... and so on...
];
const schema = [
  {
    name: "Name",
    type: "dimension",
  },
  {
    name: "Maker",
    type: "dimension",
  },
  {
    name: "Miles_per_Gallon",
    type: "measure",
    defAggFn: "avg",
  },
  {
    name: "Displacement",
    type: "measure",
    defAggFn: "sum",
  },
  {
    name: "Horsepower",
    type: "measure",
    defAggFn: "sum",
  },
  {
    name: "Weight_in_lbs",
    type: "measure",
    defAggFn: "min",
  },
  {
    name: "Acceleration",
    type: "measure",
    defAggFn: "sum",
  },
  {
    name: "Origin",
    type: "dimension",
  },
  {
    name: "Cylinders",
    type: "dimension",
  },
  {
    name: "Year",
    type: "dimension",
    subtype: "temporal",
    format: "%Y-%m-%d",
  },
];
const formattedData = await Datamodel.loadData(data, schema);
let dm = new Datamodel(formattedData);
const outputDM = dm.groupBy(
  ["Year"],
  ["Horsepower", Datamodel.AggregationFunctions.MAX],
);

Printing the output outputDM gives:

YearMiles_per_GallonDisplacementHorsepowerWeight_in_lbsAcceleration
-1980000018455147.55882352941177183512.544117647058824
315162000021.25400104.92857142857143161315.310344827586206
630522000018.714285714285715429120.17857142857143210015.125
946746000017.1455130.475186714.3125
1262106000022.70370370370370235094.23076923076923164916.203703703703702

2. sort

Syntax:  sort(sortingDetails)
parameters:
 - sortingDetails:
   required: true
   type: Array of Array
   descriptions: Sorting details based on which the sorting will be performed.

Performs sorting according to the specified sorting details.Like every other operator it doesn't mutate the current DataModel instance on which it was called, instead returns a new DataModel instance containing the sorted data.

DataModel support multi level sorting by listing the variables using which sorting needs to be performed and the type of sorting ASC or DESC.

Returns: DataModel (Returns a new instance of DataModel with sorted data)

In the following example, data is sorted by Origin field in DESC order in first level followed by another level of sorting by Acceleration in ASC order.

const Datamodel = muze.DataModel;
const data = {
  //... Cars Data as shown in above example ...
};
const schema = [
  //... Cars Schema as shown in above example ...
];
const formattedData = await Datamodel.loadData(data, schema);
let dm = new Datamodel(formattedData);
const outputDM = dm.sort([
  ["Origin", "desc"],
  ["Acceleration"], // Default value is ASC
]);

Printing the outputDM gives:

NameMakerMiles_per_GallonDisplacementHorsepowerWeight_in_lbsAccelerationOriginCylindersYear
plymouth 'cuda 340plymouth1434016036098USA8-19800000
ford mustang boss 302fordNaN30214033538USA8-19800000
plymouth fury iiiplymouth1444021543128.5USA8-19800000
amc ambassador dplamc1539019038508.5USA8-19800000
chevrolet impalachevrolet1445422043549USA8-19800000

3. calculateVariable

Syntax:  calculateVariable(params)
parameters:
 - schema:
   required: true
   type: JSON
   description : Schema of newly defined variable
 - fieldName(s):
   required: true
   type: Array
   description : Array of previous schema variable names
 - resolverFunction:
   required: true
   type: Array
   description : A function to define, how every value of the new field is generated by each
                 values of the field names, ie the previous params

Returns: DataModel (Instance of DataModel with the new field)

Creates a new variable calculated from existing variable. This method expects definition of the newly created variable and a function which resolves value of the new variable from existing variables.

Creates a new measure based on existing variables:

Example 1

const Datamodel = muze.DataModel;
const data = {
  //... Cars Data as shown in above example ...
};
const schema = [
  {
    name: "Origin",
    type: "dimension",
  },
  {
    name: "Cylinders",
    type: "dimension",
  },
  {
    name: "Horsepower",
    type: "measure",
    defAggFn: "avg",
  },
  {
    name: "Weight_in_lbs",
    type: "measure",
    defAggFn: "min",
  },
];

const formattedData = await Datamodel.loadData(data, schema);
let dm = new Datamodel(formattedData);
const outputDM = dm.calculateVariable(
  {
    name: "powerToWeight",
    type: "measure", // Schema of variable
  },
  ["Horsepower", "Weight_in_lbs"],
  (hp, weight) => hp / weight,
);

Original DataModel

OriginCylindersHorsepowerWeight_in_lbs
USA81303504
USA81653693
USA81503436
USA81503433
USA81403449

New DataModel

OriginCylindersHorsepowerWeight_in_lbspowerToWeight
USA813035040.037100456621004564
USA816536930.04467912266450041
USA815034360.043655413271245634
USA815034330.043693562481794346
USA814034490.0405914757900840

Example 2

const Datamodel = muze.DataModel;
const data = {
  //... Cars Data as shown in above example ...
};
const formattedData = await Datamodel.loadData(data, schema);
let dm = new Datamodel(formattedData);
const outputDM = dm.calculateVariable(
  {
    name: "Efficiency",
    type: "dimension",
  },
  ["Horsepower"],
  (hp) => {
    if (hp < 80) {
      return "low";
    } else if (hp < 120) {
      return "moderate";
    } else {
      return "high";
    }
  },
);

Printing outputDM gives:

NameMakerMiles_per_GallonDisplacementHorsepowerWeight_in_lbsAccelerationOriginCylindersYearEfficiency
chevrolet chevelle malibuchevrolet18307130350412USA8-19800000high
buick skylark 320buick15350165369311.5USA8-19800000high
plymouth satelliteplymouth18318150343611USA8-19800000high
amc rebel sstamc16304150343312USA8-19800000high
ford torinoford17302140344910.5USA8-19800000high

4. select

Syntax: select(conditions, config)
parameters:
 - conditions:
   required: true
   type: Object
   descriptions: An Object to govern the selection of values from the data
   default: {}
   parameters: SelectionParameters
 - config:
   required: false
   type: Object
      default: {}
   description : The configuration object to control the inclusion exclusion of a row in resultant DataModel instance.
      parameters:
     - mode:
       required: true
       type: FilteringModes
       descriptions: The mode of the selection

Comparison Operations

parameters:
  - field:
    required: true
    type: string
    descriptions: The field name to compare
  - operator:
      required: true
      type: ComparisonOperator
      descriptions: The comparison operation to be done
  - value:
      required: true
      type: SupportedDataTypes | array<SupportedDataTypes>
      descriptions: The value to be compared with

Logical Operations

parameters:
  - operator:
      required: true
      type: LogicalOperator
      descriptions: The Operation with with all the Comparison Operations are connected
  - conditions:
      required: true
      type: array<ComparisonOperations>
      descriptions: the lis of comparison operations

SupportedDataTypes: string | number | null | undefined

FilteringModes operates on the selection and rejection set to determine which one would reflect in the resultant datamodel. The Filtering modes are:

  • INVERSE
  • NORMAL
  • ALL

Note: Selection and rejection set is only a logical idea for concept explanation purpose.

Returns: DataModel (Returns an instance of DataModel with selected data according to the field names)

Example 1

const Datamodel = muze.DataModel;
const data = {
  //... Cars Data as shown in above example ...
};
const schema = [
  {
    name: "Name",
    type: "dimension",
  },
  {
    name: "Origin",
    type: "dimension",
  },
  {
    name: "Cylinders",
    type: "dimension",
  },
];

const formattedData = await Datamodel.loadData(data, schema);
const dm = new Datamodel(formattedData);
const { EQUAL } = Datamodel.ComparisonOperators;
const { AND, OR } = Datamodel.LogicalOperators;
const selectedDM = dm.select({
  field: "Origin",
  value: "Japan",
  operator: EQUAL,
});

Printing selectedDM gives:

NameOriginCylinders
toyota corona mark iiJapan4
datsun pl510Japan4
datsun pl510Japan4
toyota coronaJapan4
toyota corolla 1200Japan4

Example 2

const Datamodel = muze.DataModel;
const data = {
  //... Cars Data as shown in above example ...
};
const schema = [
  {
    name: "Name",
    type: "dimension",
  },
  {
    name: "Origin",
    type: "dimension",
  },
  {
    name: "Cylinders",
    type: "dimension",
  },
];

const formattedData = await Datamodel.loadData(data, schema);
const dm = new Datamodel(formattedData);
const { EQUAL } = Datamodel.ComparisonOperators;
const { AND, OR } = Datamodel.LogicalOperators;
const selectedDM = dm.select({
  conditions: [
    { field: "Origin", value: "Japan", operator: EQUAL },
    {
      conditions: [
        { field: "Cylinders", value: "3", operator: EQUAL },
        { field: "Cylinders", value: "6", operator: EQUAL },
        { field: "Cylinders", value: "8", operator: EQUAL },
      ],
      operator: OR,
    },
  ],
  operator: AND,
});

Printing selectedDM gives:

NameOriginCylinders
mazda rx2 coupeJapan3
maxda rx3Japan3
toyota mark iiJapan6
toyota mark iiJapan6
datsun 810Japan6

5. project

Syntax: project(projField, config)
parameters:
 - projField:
   required: true
   type: Array<(string|Regexp)>
   descriptions: An array of column names in string or regular expression.
 - config:
   required: false
   type: Object
      default: {}
   description : The configuration object to control the creation of new DataModel.
      parameters:
     - mode:
       required: true
       type: FilteringModes
       descriptions: Mode of the projection

This is functional version of projection operator. Projection is a column (field) filtering operation. It expects list of fields name and either include those or exclude those based on FilteringMode on the resultant dataModel. It returns a function which is called with the DataModel instance on which the action needs to be performed.

Projection expects array of fields name based on which it creates the selection and rejection set. All the field whose name is present in array goes in selection set and rest of the fields goes in rejection set.

FilteringModes operates on the selection and rejection set to determine which one would reflect in the resultant datamodel.

Note: Selection and rejection set is only a logical idea for concept explanation purpose.

Returns: DataModel (Returns an instance of DataModel with project data according to the field names)

Example:

const Datamodel = muze.DataModel;
const data = {
  //... Cars Data as shown in above example ...
};
const schema = [
  {
    name: "Name",
    type: "dimension",
  },
  {
    name: "Origin",
    type: "dimension",
  },
  {
    name: "Cylinders",
    type: "dimension",
  },
];

const formattedData = await Datamodel.loadData(data, schema);
const dm = new Datamodel(formattedData);
outputDM = dm.project(["Name"], { mode: Datamodel.FilteringModes.INVERSE });

Printing outputDM gives:

OriginCylinders
USA8
USA8
USA8
USA8
USA8

6. splitByRow

Syntax: splitByRow(fields)
parameters:
 - fieldNames:
   required: true
   type: Array<(string)>
   descriptions: An array of column names in string.

Returns: DataModel (Returns an array of instances of DataModel with split according to the field names)

Example 1 This is the method that is used to split into groups of unique combinations of the values of the fields. For example : If for the cars data we have been using in this section, if we split by Origin, it will be split into an array of three datamodels, each for USA, Japan and European Union

const Datamodel = muze.DataModel;
const data = {
  //... Cars Data as shown in above example ...
};
const schema = [
  {
    name: "Name",
    type: "dimension",
  },
  {
    name: "Acceleration",
    type: "measure",
    defAggFn: "avg",
  },
  {
    name: "Origin",
    type: "dimension",
  },
  {
    name: "Cylinders",
    type: "dimension",
  },
];

const formattedData = await Datamodel.loadData(data, schema);
const dm = new Datamodel(formattedData);
const outputDm = dm.splitByRow(["Origin"]);
for (let i = 0; i < outputDm.length; i++) {
  printDM(outputDm[i]);
}

Note: printDM is a utility function to render datamodel on a webpage for demonstration putpose only.

The output gives 3 datamodels as shown below:

NameAccelerationOriginCylinders
citroen ds-21 pallas17.5Europe4
volkswagen 1131 deluxe sedan20.5Europe4
peugeot 50417.5Europe4
audi 100 ls14.5Europe4
saab 99e17.5Europe4
NameAccelerationOriginCylinders
chevrolet chevelle malibu12USA8
buick skylark 32011.5USA8
plymouth satellite11USA8
amc rebel sst12USA8
ford torino10.5USA8
NameAccelerationOriginCylinders
toyota corona mark ii15Japan4
datsun pl51014.5Japan4
datsun pl51014.5Japan4
toyota corona14Japan4
toyota corolla 120019Japan4

Example 2

Similarly if the data can me split by unique combinations of more than one field values. Considering, the following sample, when the data id split by Origin and Cylinders, the number of data models generated is 9 because the unique groups formed by every values of Origin and Cylinders is 9:

const Datamodel = muze.DataModel;
const data = {
  //... Cars Data as shown in above example ...
};
const schema = [
  {
    name: "Name",
    type: "dimension",
  },
  {
    name: "Acceleration",
    type: "measure",
    defAggFn: "avg",
  },
  {
    name: "Origin",
    type: "dimension",
  },
  {
    name: "Cylinders",
    type: "dimension",
  },
];

const formattedData = await Datamodel.loadData(data, schema);
const dm = new Datamodel(formattedData);
const outputDm = dm.splitByRow(["Origin", "Cylinders"]);
for (let i = 0; i < outputDm.length; i++) {
  printTable(
    outputDm[i].getData().data,
    ["Name", "Acceleration", "Origin", "Cylinders"],
    { rowLimit: 1 },
  );
}

Note: printTable is a utility function to render datamodel on a webpage for demonstration putpose only.

NameAccelerationOriginCylinders
chevrolet chevelle malibu12USA8
NameAccelerationOriginCylinders
audi 500015.9Europe5
NameAccelerationOriginCylinders
mercedes-benz 280s16.7Europe6
NameAccelerationOriginCylinders
toyota mark ii13.5Japan6
NameAccelerationOriginCylinders
mazda rx2 coupe13.5Japan3
NameAccelerationOriginCylinders
chevrolet vega 230015.5USA4
NameAccelerationOriginCylinders
plymouth duster15.5USA6
NameAccelerationOriginCylinders
toyota corona mark ii15Japan4
NameAccelerationOriginCylinders
citroen ds-21 pallas17.5Europe4